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ABSTRACT 

While supervisory research focuses on trainee 
performance in counseling situations as the primary outcome 
criterion, few instruments have been developed for evaluating 
behavior in the supervision interview. To develop a scale that 
reflects critical factors in the supervisory relationship, the 
Supervisor Personal Reaction Scale (SPRS) and the Trainee Personal 
Reaction Scale (TPRS) were factor analyzed using 140 and 141 
questionnaires, respectively. The results indicated that for both the 
SPRS and TPRS, 12 items of the original 32-item questionnaires 
defined three relatively independent factors. Since the items within 
each factor for both the SPRS and TPRS were conceptually parallel, 
the three subscales were labelled in the same way for each scale. The 
subscales, each with four items apiece, were labelled Evaluation of 
Other, Evaluation of Self, and Level of Comfort. The new 12-iten) 
questionnaires were named the SPRS-Revised (SPRS-R) and the 
TPRS-Revised (TPRS-R). The new scales provide useful measures for 
research and for the training of supervisors. (Author/'JAC) 
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Abstract 

The Supervisor Personal Reaction Scale (SPRS) and the Trainee Personal 
Reaction Scale (TPRS) were factor analysed using 140 and 141 questionnaires, 
respectively. The results indicated that for both the SPRS and TPRS, 12 items 
of the original 32-item questionnaires defined three relatively independent 
factors. Since the items within each factor for both the SPRS and TPRS were 
conceptually parallel, the three subscales were labelled in the same way for 
each scale. The subscales, each with 4 items apiece, were labelled as 
follows: (a) Evaluation of Other, (b) Evaluation of Self and (c) Level of 
Comfort. The new 12 item questionnaires were named the SPRS - revised 
(SPRS-R) and the TPRS - revised (TPRS-R). The application of the subscales to 
previous research is presented and in^lications for the SPRS-R and the TPRS-R 
in research and training in supervision are discussed. 
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Supervision Satisfaction 

With few exceptions, researchers of supervision have depended primarily 
on counselor rating scales to evaluate supervision outcomes. Two 
circumstances support this situation. First, most supervisory research 
focuses on trainee performance in a counselins situation as the primary 
outcome criterion (Hansen, Robins and Grimes, 1982; Lambert, 1980). 
Therefore, counselor rating scales such as the counselor Evaluation Rating 
Scale (CBRS; Myrick & Kelly, 1971), the Counselor Rating Form (CRF; Barack & 
Lacrosse, 1975) and the Barrett-Leonard Relationship Inventory 
(Barrett-Leonard, 1962) are useful and relevant to these types of experimental 
questions . 

Second, there are few psychometrically validated instruments that have 
been developed exclusively for evaluating supervisor and trainee behavior in 
the supervision interview. Because of the paucity of supervision instruments, 
researchers* with a few notable exceptions (Doehrmann 1976; Heppner and 
Handley, 1982; Holloway, 1979; Holloway & Wampold, 1983; Worthington & Roehlke 
1979), have used instruments to assess the supervision relationship that have 
actually been developed for and validated on the counseling relationship. For 
instance, Carkhuffs Scales (Carkhuff , 1969) are very frequently used to 
indicate the supervisor's delivery of facilitative conditions in supervision 
(e.g. Dalton & Sundblad, 1976; Lambert & Beier, 1974; Pierce, Carkhuff f & 
Berenson, 1967; Pierce & Schauble, 1970, 1971). Scales such as the CERS and 
the CRF are aplied to determine supervisors* attractiveness to trainees (a-g-i 
Dodenhoff, 1981; Heater, Weit^, Anchor & Roback, 1976). "Satisfaction" 
ratings such as the Interview Rating Scale (Anderson & Anderson, 1962) and the 
Barrett-Leonard Relationship Inventory (1962) are employed to indicate 



ERIC 



4 



Supdrvision Satisfaction 
4 

traineas* and superviaora* aatiaf action with tha suparvision intarviaw (a.R., 
Lennona & Lannins, 1979). Althoush supervisor's delivery of facilitativa 
conditions, attractiveness to trainee, and trainee's performance mi&ht be 
relevant to evaluation in supervision, other evaluation measures, designed 
specifically for the supervisory context, irarrant consideration. 

Doehrman (1976), in a phenomenolosical study of parallel processes in 
supervision, developed a paper-and-pencil Likert-type questionnaire to rate 
both trainee's and supervisors* perceptions of the supervisory relationship. 
Her scales, althou&h sensitive and revealing in her own study (when used in 
connection with clinical interview data) have not been further developed 
pflychofiieirically. Another instrument developed for the supervisory context is 
the Supervisor Questionnaire (SQ; Uorthin&ton and Roehlke, 1979) which 
measures trainees' judgement of supervisors* performance in supervision. The 
questionnaire includes 42 supervisor behaviors rated on 5-point Likert-type 
rating scales. Factor analysis of the SQ revealed two primary factors that 
were labelled Evaluation and Support. Heppner and Handley (1982) used the SQ 
to measure supervisors' judgment of their own sup«rvisory behaviors. 

We were interested in developing a scale that reflects critical factors 
in the supervisory relationship. However, in addition to evaluating the 
supervisors' behaviors from both participants point of view, as was the case 
in Doehrman (1976) and Uorthington and Roehlke (1979), we included evaluative 
dimensions that reflect the trainees* behavior as a trainee. By including 
both supervisors' and trainees* judgments of trainees behaviors in the 
supervision interview we have deviated from two traditional assumptions: (a) 
that the supervisor is solely responsible for directing the supervisory 
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interview and is the focus of evaluation within the supervision interview, and 
(b) that the trainee is primarily responsible for his/her behavior as a 
counselor and therefore is evaluated in the counselins interview. Our 
instnintent also differs from the SQ in that it measures reactions to a 
particular interview rather than reactions at the termination of the 
supervisory relationship. We felt that focusins on the supervision 
interaction itself would be helpful in understanding effective supervisor and 
trainee connunication strategies within the context of supervision. 

Our development and use of ttro scales - the Supervisor Personal Reaction 
Scale (SPRS) and the Trainee Personal Reaction Scale (TPRS) emerged from our 
empirical study of supervisory behavior. We wanted an instrument that: (a) 
was a self -report, paper and pencil instrument; (b) had a completion time of 
10 minutes or less; (c) described dimensions of the supervisory interview and 
supervisory relationship; (d) could be administered repeatedly to the same 
participants after different supervision interviews; (e) could measure 
dimensions of satisfaction of supervision from t^e supervisor's and trainee's 
points of view; and (f) could be applied for both research and training 
purposes. The present paper reports the development, psychometric structure 
and application of SPRS and TPRS. 

METHOD 

Development of SPRS and TPRS 

The SPRS and TPRS resulted from an adaptation of the Therapist Personal 
Reaction Scale (ThPRS) and Client Personal Reaction Scale (CPRS) reported by 
Ashby, Ford, Guemey and Guemey (1957). The original ThPRS and CPRT were 
constructed in an identical fashion each with two subscales, named Wegative or 
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DAfenaive and Positive. All 35 itams of tha IhPRS and 40 itoms of tha CPRS 
wara naaaurad on a Likart-typa scale of 1 to 5. Total scores of each subscala 
are coiBputed. The "Nesative** subscale of ThPRS tias intended to reflect the 
therapist's negative reactions to the therapy interview and the client. The 
**Po8itive" subscale reflects the therapist's feelings of progress, achievement 
and accomplishment with the client in therapy. The CPRS "Defensive" subscale 
indicates the client's negative reaction to the therapist and therapy. The 
"Positive" subscale indicates client feelings of progress, achievement and 
accomplishment. The SPRS and TPRS adapted the items of the ThPRS and CPRS, 
respectively and omitted those items that intuitively appeared irrelevant to 
the supervisory context. The first form of the scale included 32 items, with 
bo«-.h negative and positive valence, measured on a 5~point Likert-type scale. 
The instructions for completing the questionnaire we.-e as follows: 
During supervision, trainees (supervisors) have many different 
feelings and reactions. These reactions are sometimes negative, 
sometimes positive, and sometimes mixed. Having varied feelings and 
reactions toward your supervisor (trainee) is not undesirable as 
long as you recognize and understand them. I am interested in 
learning What feelings trainees (supervisors) have toward 
supervision sessions. There are five possible answers to each item 
in the questionnaire. They are: 1. Hot characteristic of my 
present feelings. 2. Slightly cliaracteristic of my present 
feelings. 3. Moderately characteristic of my present feelings. 4. 
Quite characteristic of my present feelings. 5. Highly 
chartEicteristic of my present feelings. Put a circle around tha 
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anatror wout repreaentativa of your PRBSBNT faalinga about tha 
aupervlalon aeaalon you juat participatad in. Thank you for your 
cooparation. 

Bacauae it iraa not obvioua that the same dafansiva, nagativa and poaitiva 
dimanaiona might be present in the scales When used in a supervisory context 
and becauae a factor analysis had not been performed on the original ThPRS and 
CFRS, it appeared that such an analysis would be an iinportant and necessary 
aspect of understanding the usefulness of the scales. 
Validation Saiaples and Procedures 

The SPRS and TPSS were used in two different studies during a period of 
three years (Holloway, 1979; Holloway & Uainpold, 1983). The first study 
(Hollo«ray, 1979) was an analogue design with beginning level supervisors and 
traineaes. The second study was carried out in a naturalistic setting over a 
period of two quarters in different academic years. Bxperienced doctoral 
level supervisors and beginning masters level trainees participated in the 
later study. All supervisors had a minimum of a didactic courae and 
supervised practicum in supervision. All trainees had at leaat a prepracticum 
interviewing course. The participanta for each study were from different 
counseling programa. Data was collected from a total of 22 supervisors (14 
women, 8 men) and 66 trainees (42 women, 24 men). The total number of 
interview was 150. There were 140 SPRS and 141 TPRS questionnaires used in 
the data analysis. The remaining questionnair * a were incompletely filled out 
and eliodnated from the data set. 

The Analogue Study . Doctoral and masters students in counseling and 
guidance were assigned supervisor and trainee roles, respectively, for a 
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sinulated supervision session « All participants viewed a 10«S minute 
videotape consisting of five senements of an actual client-counselor 
interaction. The counseling videotape provided an appropriate cotnmon topic of 
discussion for the subsequent supervision interviews. Bach supervisor ma 
randomly assigned two trainees for 45 minute individual supervision sessions* 
Participants were instructed to (a) focus their supervision disccussion on the 
client in the videotape; (b) act as if the client in the videotape had been 
referred to the trainee and would see the trainee the next week for 
counseling; and (c) behave as if both supervisor and trainee would have the 
opportunity to meet again the following week. All interviews were 
audio- taped. At the completion of the interview, supervisors and trainees 
independently completed the SPRS and TPRS, respectively, 

The Naturalistic Study . Supervisors were asked to audiotape sessions 3, 
6 and 9 of their regular weekly one-hour supervision sessions with each of 
their counaolor trainees. The distribution of recorded interviews across the 
12-week quarter helped to produce a final sample that would be representative 
of different stages of the supervisory relationship as it progressed during 
the quarter. At the conslusion of each interview the supervisor and trainee 
indepettdently completed the SPRS and TPRS, respectively, and deposited them in 
a return box. The supervisors and trainees were asked not to discuss their 
responses with each other at any time during the quarter and were assured that 
their ratings would in no way influence their evaluation in the practicum 
course. 

Analyses and Results 

Se lection of Items 

A preliminary factor analyses with orthogonal rotation on the 32 items of 
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the SPRS and TPRS irae conducted to invest isate the factorial structure of the 
SPRS and the TPRS. As reported elsewhere (Holloway and Wampold, 1983), this 
analysis revealed three primary factors for each scale: Factor 1, Evaluation 
of Other; Factor 2, Evaluation of Self; and Factor 3, Level of Comfort. 

The objective of this analysis was to select items from the SPRS and the 
TPRS to form revised scales (SPRS-R and TPRS-R) that would have reliable and 
relatively independent subscales. As with any factor analysis with orthogonal 
rotations the three factors from the preliminary factor analysis yielded 
factors that were uncorrelated. The usual practice is then to select those 
items that load hishest on the first factor to comprise the first subscore, 
those items that load highest on the second factor to form the second 
subscale, and so forth. Unfortunately, because items load on more than one 
factor, the subscales can easily be highly correlated even though factors are 
independent (Gorsuch, 1974). Ue chose a strategy to select items for the 
scales ihat would attenuate the inter-subscale correlations.^ As suggested 
by Gorsuch (1974), those items that loaded most heavily on more than one 
factor were eliminated, leaving items that loaded primarly on one factor. The 
items selected for each subscale are shown in Table 1. 



Insert Table 1 About Here 



To determine whether the subscales had desirable properties, a number of 
analyses were conducted. First, Cronbach*s alpha was calculated for each 
subscale to establish internal consistency (Hunnally, 1967). These values are 
found in Ihe diagonals of Table 2. The average alpha for the six subscales 
was .78. 
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Insert Tab la 2 about Kara 



Second, the Inter-subscale correlations ware calculated and appear In 
table 2. The average Inter-subscale correlation «#as .45. Although this value 
is not negligible, it is substantially lower than the average inter-subscale 
correlations for factor analystic studies that do not select items based on 
their relative contributions to factor. Finally, a principle component factor 
analysis tfith varimax rotation (Morrison, 1976) was conducted on the items 
comprising the SPRS-R and the TPRS-R. As can be seen in Table 3, the items in 
each scale loaded most heavily on the corresponding factor, as would be 
expected by using the strategy described above. 



Insert Table 3 About Here 
Discussion 

The Validation Studies 

In the analogue study (Holloiray, 1979), the SPRS and TPRS were used as 
global scales resulting in one total score. There were no significant 
differences between the participants* ratings of the interviews, although 
supervisors informally reported to the experimenter that they experienced 
different levels of frustration and adequacy with different trainees, it 
seemed that the overall score was not able to reflect these different 
reactions. Recognition that a supervisor or trainee could feel very positive 
about their own performance even though there were uncomfortable issues to 
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deal with and/or they judged their counterpart negatively in the interview, 
led to our interest in identifyins relevant dimensions within the overall 
scale. In the naturalistic study (Holloway & Wampold, 1983), the factor 
scores of the scales were used as criterion variables in a multiple regression 
with patterns of supervisor and trainee interchange as the predictor 
variables. Ttie use of separate factors within the scale proved valuable in 
identifying specific supervisory exchanges that corresponded to positive and 
negative reactions by the participants. For instance, the Level of Comfort 
subscale score for both participants increased when defensiveness or criticism 
on either the part of the supervisor or trainee was present in the interview. 
Tlie Evaluation of Self subscale for both participants increased When the 
supervisor encouraged the trainee to expand on their ideas. The supervisor's 

Evaluation of Other subscale score decreased When trainees were defensive or 

2 

used excessive positive social esK>tional behavior. 
The Subscales 

The SPRS-R and TPRS-R have a number of advantages over the SPRS and 
TPRS. Reliable and relatively independent subscales have been developed that 
have conceptually clear meanings. The subscales can be completed easily and 
score on the subscales can be calculated simply by the supervisor and trainee, 
by the researcher or the supervisor educator. It should be mentioned that 
validation studies of the SPRS-R and the TPRS-R were conducted using total 
scores or factor scores. Therefore, it remains to be shown conclusively that 
the subscales of the SPRS and TPRS will have equally useful application. 

The items of each factor in the SPRS-R and TPRS-H appear to group 
together in a meaningful and consistent way. Items that reflect self 
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•valuation, other evaluation, and level of comfort emerged as factors In each 
scale. The Items themselves appear to tap Interpersonal attributes, social 
interaction and performance judgments rather than specific techniques or 
strategies. Unlike the revised Worthington and Roehlke (1979; Heppner & 
Handley, 1982) questionnaire, the scales do not deal with the incidence of 
particular supervisory strategies for evaluation. The SPRS-R and TPRS-R 
factors provide a gauge for the climate of a particular interview as seen from 
the different perspectives of the supervisor and the trainee. 

Bach of the subscales. Evaluation of Other, Evaluation of Self and Level 
of Comfort can be explained within the context of the supervisory 
relationship. The subscale. Evaluation of Other, appears salient not only 
because of one's natural reaction to judge one's counterpart in any social 
interaction, but also because of the component of evaluation in supervision. 
Both participants are aware that their skills are being judged; the trainee to 
demonstrate competency as a counselor, the supervisor to be a skillful 
teacher. The Evaluation of Self subscale of the SPRS-R may reflect the 
supervisors' implicit responsibility to guide the interview (i.e., to instruct 
and to consult) and thus the need for self examination of their own 
supervisory behaviors that will facilitate the trainee's learning. The 
supervisors' Level of Comfort Subscale on the other hand, may be tapping the 
supervisors on-going judgment of their performance in the interview. The 
TPRS-R sbuscale. Level of Comfort most likely reflects the trainees atrareness 
of the supervisors explicit obligation to evaluate the trainees' performance 
and the supervisors' superordinate position in the relationship. The TPRS-R 
subscale. Evaluation of Self, reflects the trainees' judgments of their own 
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raaponsaa to the superviaora. The characteristics of this aubacale raise an 
intarastins iaaue. As mentioned earlier, trainee instruction and evaluation 
has focused on the traineea* performance in the counselins ralationahip. 
There are very few empirical studies that examine trainee behavior in the 
superviaory interview as a relevant outcome event. Not aurprisinfly, then, 
there are no explicit suidelinea for appropriate or effective trainee behavior 
in this context. Clearly, however, supervisors make judgments about traineea 
baaed on the trainees* aupervisory responses; behavior that is distinct from 
that directly representative of their counselins performance. For instance, 
the psychodynamic model of supervision emphasizes the isq^ortance of trainee 
behavior in supervision as a reflection of their countertransference in the 
counselins relationship (Ekstein & Wallerstein, 1974). It would seem that the 
TPRS-R subscale Evaluation of Self offers a now dimension to supervision 
evaluation. 

Application to Research and Training 

The factor analysis of the SPRS-R and TPRS-R has resulted in a 
questionnaire that is brief, reliable, and measures three dimensions of 
satisfaction that appear conceptually relevant to the supervision process. In 
research and in the training of supervisors, the scales misht be used (a) to 
identify supervisory behaviors that promote or detract from the relationship; 
(b) to examine the trainee* a aupervisory behavior as an outcome measure; (c) 
to compare supervisor's and trainee's perceptions of the supervisory 
interview; (d) to compare supervisor's ratings of self and trainee's rating of 
other and vice versa; and (e) to track the change in satisfaction with the 
aupervisory relationship across time from aupervisor'a and trainee* a 
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perspectives. 

As evidenced by two recent volumes of The Counseling Psvcholoitlst 
(Supervision Z, Vol. 1982; Supervision ZZ, In press) supervision Is currently 
attractlns considerable attention. Evaluation Instruments that measure 
supervisory behaviors, attitudes, and perceptions will be a necessary 
component of the research and practice of supervision. Hopefully, the SPRS-R 
and TPRS-R will contribute to these endeavors. 
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Footnotes 



An alt«m«ttt stratttsy and one which yields Independent subscales Is to 
calculate factor scores (Gorsuch, 1974; Morrison, 197^). Factor scores take 
Into account Items* relative loadings on each factor. Although factor scores 
are useful (e.g.. Holloway. & Wampold. 1983). they are difficult to calculate, 
particularly When applied to data other than that used in the factor 
analysis. Furthermore, the adequacy of factor scores shows degradation upon 
crofis- validation Whereas, the method we used is quite robust (Gorsuch, 1974). 
Our objective was to generate subscales that could be used easily in research 
and practice. 

^The subscales of the SPRS and TPRS were predicted by several other 
supervisor and trainee verbal behaviors as reported in Holloway and Wampold 
(1983). 
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Table 1 

Listing of Subscale Items for the SPRS-R and the TPRS-R 



SPRS-R 



Subscale 1: Evaluation of Trainee 

1. I have a warm, friendly reaction to this trainee. 

2. I disagree with this trainee about some basic matters. 

3. I was very absorved In what this trainee was saying. 

4. I'm glad this particular trainee was assigned to me. 

Subscale 2: Evaluation of Self as Supervisor 

5. It was hard to know how to respond to this trainee In a helpful 

6. I usually found significant things to respond to In what the 
trainee said. 

7. I think I did a pretty conpetent job with this trainee. 

8. I felt pretty Ineffective with this trainee. 

Subscale 3; Level of Comfort In the Interview 

9. I was seldom in doubt about what the trainee was trying to say. 

10. Sometimes I felt pretty frustrated during the Interview. 

11. Sometimes I got pretty tense during the Interview. 

12. I think we had a pretty relaxed, understanding kind of 
relationship during the Interview. 



TPRS-R ' 




Subscale 1: Evaluation of Supervlson 

1. I was eager to hear what ay supervisor had to say. 

2. My supervisor's attitude gave me hope that I can really get 
something out of supervision. 

3. Many of the things my supervisor said reaVly 'hit the 
nal 1 -on-the-head. 

4. I gained more respect for supervision as a result of my 
experience with this supervisor. 

Subscale 2: Evaluation of Self as Trainee 

5. Sometimes the supervisor seemed to twist around the things I 
said to mean something different than what I Intended. ^ 

6. Sometimes after the supervisor said something I just couldn t 
think of any response. 

I felt my supervisor wanted me to come to some conclusions 
about the client, but I didn't know exactly what. 
8. I sometimes felt like I was being put-on-the spot. 
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table 1 cont'd 

Subscale 3: Level of Comfort In the Interview 

9. At times, I hesitated to tell my supervisor what I was really 
thinking. 

10. I got Irritated at some of my supervisor's remarks. 

11. I don't know exactly why, but I felt nervous during the 
interview. 

12. I sometimes resented my supervisor's attitude toward me. 



Note . The SPRS-R and TPRS-R should present the subscale items In 
random order and not as numbered In this table. 
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Table 2 

Correlations among Subscales and Internal Consistencies 

for the SPRS-R and TPRS-R 



SPRS-R 



Subscale 1 


Subscale 1 
(.83) 




Subscale 2 
.50 


Subscale 3 
.54 


Subscale 2 






(.72) 


.49 


Subscale 3 








(.78) 


TPRS-R 




Subscale 1 




Subscale 2 


Subscale 3 


Subscale 1: 


(.89) 




.31 


.42 


Subscale 2: 






(.71) 


.42 


Subscale 3: 








(.76) 


Note. Measures 
In parentheses. 


of Internal 


consl stency 

• 


(Cronbach's 


alpha) are found 



2Z 

ERIC 



Table 3 



Orthogonal Rotated Factor Analyses for the SPRS-R and TPRS-R 



SPRS-R 



Items 

Subscale 1 
1 
2 
3 
4 



Factor Loadings 



Factor 1 

.79 
.43 
.61 
.89 



Factor 2 



Factor 3 



Subscale 2 
5 
6 
7 
8 



61 
44 
60 
67 



Subscale 3 
9 
10 
11 
12 
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.59 
.48 
.71 
.71 



TPRS-R 



Items^ 

Subscale 1 
1 
2 
3 
4 



Factor 1 

.65 
.80 
.81 
.92 



Factor Loadlng^^ 
Factor 2 



Factor 3 



Subscal e 
1 
2 
3 
4 



51 
44 

57 
78 



Subscal e 
1 
2 
3 
4 



.71 
.69 
.64 
.47 



Note^. Factor Loadings less than .35 were omitted 
Noteb. Items are listed In Table 1. 
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